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ABSTRACT 


Humans share a universal and fundamental set of emotions which are exhibited 
through consistent facial expressions. Recognition of human emotions from the imaging 
templates is useful in a wide variety of human-computer interaction and intelligent 
systems applications. However, the automatic recognition of facial expressions using 
image template matching techniques suffer from the natural variability with facial 
features and recording conditions. In spite of the progress achieved in facial emotion 
recognition in recent years, the effective and computationally simple feature extraction 
and classification technique for emotion recognition is still an open problem. Image 
pre-processing and normalization is significant part of face recognition systems. 
Changes in lighting conditions produces dramatically decrease of recognition 
performance. In this paper, the image pre-processing techniques like K-Nearest 
Neighbor, Cultural Algorithm and Genetic Algorithm are used to remove the noise in 
the facial image for enhancing the emotion recognition. The performance of the pre- 
processing techniques are evaluated with various performance metrics. 
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1. INTRODUCTION 


Emotion recognition 1s being actively explored in Computer Vision research. With the recent 
rise and popularization of Machine Learning [1] and Deep Learning [2] techniques, the potential 
to build intelligent systems that accurately recognize emotions became a closer reality. 
However, this problem is shown to be more and more complex with the progress of fields that 
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are directly linked with emotion recognition, such as psychology and neurology. Micro- 
expressions, electroencephalography (EEG) signals, gestures, tone of voice, facial expressions, 
and surrounding context are some terms that have a powerful impact when identifying emotions 
in a human [3]. When all of these variables are pieced together with the limitations and problems 
of the current Computer Vision algorithms, emotion recognition can get highly complex. Face 
detection techniques are able to create bounding boxes that delimit detected faces, which are 
the desired regions of interest (ROIs) for a conventional FER system. This task is still 
challenging, and it is not guaranteed that all faces are going to be detected in a given input 
image. This is especially true when acquiring images from an uncontrolled environment, where 
there may be movement, harsh lighting conditions, different poses, great distances, among other 
factors [4]. 


Human emotion detection is implemented in many areas requiring additional security or 
information about the person. It can be seen as a second step to face detection where we may 
be required to set up a second layer of security, where along with the face, the emotion 1s also 
detected. Human emotions can be classified as: fear, contempt, disgust, anger, surprise, sad, 
happy, and neutral. These emotions are very subtle. Facial muscle contortions are very minimal 
and detecting these differences can be very challenging as even a small difference results in 
different expressions [4]. Also, expressions of different or even the same people might vary for 
the same emotion, as emotions are hugely context dependent [7]. 


Through this paper, an optimized image pre-processing technique is proposed to enhance 
the quality of the face image for improving the classification of the human emotion using KNN 
and Cultural algorithm. KNN is one of the most classification algorithm whereas CA used to 
take optimized K value for the KNN. 


1.1. Background Study on Image Enhancement and Noise Removal 


Recent times, image enhancement happens to be a factor of several image that 1s important and 
applications of computer vision. Image enhancement requires getting a graphic and enhancing 
it visually, commonly if you take advantageous asset of a reaction to stimuli that are visual. 
Sequences of enhancement methods are extensively utilized to enable the introduction of a 
remedy for computer image issues. A most of these techniques focused on low illumination or 
magnification that is high problems connected with noise persist. In an Image Enhancement 
technique [9], it is observed that why, noise removal is still an image that is important task. 
Image noise presents excessive or information that is undesired can happen through the image 
capture, acquisition, processing and transmission, and can even be dependent or in addition to 
the image content. The noise can be modeled with either a Gaussian, Uniform or Salt-and- 
Pepper distribution [10] in typical images. A filter is one thing that attenuates or enhances 
frequencies that are particular to visualize when you look at the frequency domain. Image filters 
are primarily applied to alter the characteristics of images such as colors, size, shading, and 
others. There is variety of use in image processing operations implemented with this filtering 
includes smoothing, sharpening, and edge enhancement. 


2. RELATED WORKS 


Bhattacherjee, Payal, and M. M. Ramya [11] utilized Fuzzy Logic for contrast development 
since it relates to uncertainties in image acquisition. New contrast development of facial images 
predicated on new enhanced fuzzy set theory 1s suggested. This paper geared towards adaptive 
enhancement that is fuzzy of facial images for pain evaluation. Every emotion is connected 
with certain mix of action units. 
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Amrapur, Basavaraj [12] utilized snake or contour that is active when it comes to segment 
of facial facts. The important facial characteristics gathered from images are used to verify (or) 
identify the human face. 


Rathika, N., P. Suresh, and N. Sathya [13] explained a novel approach LS enhancement 
method utilizing Column-Wise Sliding Neighbourhood Operations (CWSNO) and General 
Sliding Neighborhood Operation (GSNO), and pulls the Principal Components Analysis (PCA) 
features with three straight ways such as median, mean and mode that are then categorized with 
Minimum Distance (MD) classifier using LOOCV (Leave One Out Cross Validation) of Re- 
Sampling Method (R-SM) to identify the faces. 


Zhilali et. Al [14] have proposed a technique to extract facial features based on the Local 
Binary Pattern (LBP) with lighting affect. In the LBP method, it alters the face traits by 
acquiring the binary type through the thresholding result around pixels or commonly called 
neighbors pixels. 


Oloyede et al [15] introduced a new evaluation method for enhancing the performance of 
the Meta Heuristic-based Image Enhancement technique. Basically, the authors employed this 
new evaluation function together with meta heuristic-based optimization methods so that you 
can select instantly the greatest enhanced face image predicated on a linear combination of 
various key measure that is quantitative. 


Oulefki et al [16] aimed the effectiveness of the Contrast Limited Adaptive Histogram 
Equalization (CLAHE) protocol for face preprocessing. Fuzzy Inference System is applied to 
improve the annoyance of non-uniform illumination of face images in a targeted and precise 
manner. As a result the illumination problem is occurred due to low-light. Initially, the input 
face image is divided in to two sub-regions that are equal. Eventually, the amount of brightness 
in every sub-region plus in the face that is whole utilized for dynamic decision of whether or 
not to normalize. 


Yang et al [17] suggested a GPU based Region Covariance Filter (RCF)-Retinex, that could 
boost region covariance filter with CUDA. It expansion is really feasible to utilize CUDA to 
parallel the covariance that is region because of its successive convolution operations, therefore 
it may receive the illumination image quickly. 


Ilyas et al [18] recommended a Face Recognition (FR) system divided in to three steps: the 
face viola-Jones algorithm, facial image enhancement utilizing Adaptive Histogram 
Equalization algorithm (AHE), and show learning for classification. 


Ravinaik, D., et al [19] introduced a concept that is novel modify the current power law 
transform to boost excellence of face region for better identification. The Double Density Dual 
Tree Discrete Wavelet Transforms (DDDTDWT) is employed to extract features. The 
Euclidian distance (ED) is employed to suit the acquired popular features of database and test 
face images to calculate performance variables. 

Bendyjillali, Ridha Ilyas, et al [20] presented a Face Recognition (FR) system divided in to 
three steps: the face that is viola-Jones algorithm, facial image enhancement using Modified 
Contrast Limited Adaptive Histogram Equalization algorithm (M-CLAHEB), and show learning 
for classification. 


3. PRE-PROCESSING TECHNIQUE FOR FACE RECONGITION 
3.1. K Nearest Neighbor Technique 


The K-Nearest Neighbor algorithm (K-NN) is a non-parametric method used for classification 
and regression in pattern recognition. Both in cases, the input is made from the K training are 
closest examples in the feature space. K-NN is a kind of instance-based learning. A test example 
classification is shown when look at the figure 1. Look at the test sample is a big dot within the 
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circles that will be classified both into the high grade of triangles or even the next class of 
squares. If K=5 (dashed line circle) it really is designated into the second class you can find 3 
squares and 2 triangles inside that circle. If K=3 (solid line circle) it really is assigned into the 
second class because here | triangle and 2 squares within that circle. It could be useful in the 
event that weight contributions regarding the neighbours are believed considering that the 
nearer neighbours contribute significantly more than the distant ones. As an example, in a 
common weighting, individual neighbour is assigned to a weight of 1/d if dis the exact distance 
into the neighbour. The shortest distance among any two neighbours is often a straight line, the 
exact distance is recognized as Euclidean distance [19]. The limitation regarding the K-NN 
algorithm could it be’s responsive to the local configuration of data. The entire process of 
transforming the input data to a collection of features is recognized as Feature extraction. In 
Feature space, extraction is taken put on raw data before you apply KNN algorithm. 


a= = 
—_ 7 





# ™ 
~ 
Po a . 
‘ 
/ 
; i : 
e “~° 
I 
. 
\ / 
' A / 
‘ / 

‘ a 
\. Pea 
Wie 
K=5 


Figure 1 A K-Nearest Neighbor 


3.2. Cultural Algorithm 


The CA is a dual inheritance that defines evolution in human culture at both the macro- 
evolutionary level, which requires place inside the belief space, as well as the micro- 
evolutionary level, which happens in the population space. CA is made from a population that 
is social a belief space. Connection with individuals selected through the population space 
because of the acceptance function is employed to build problem solving knowledge resides 
when look at the belief space. The belief space manipulates and stores the ability acquired 
through the connection with individuals when look at the population space. The evolution can 
be controlled by this knowledge regarding the population component in the form of the 
influence function. Because of this, CA can offer a mechanism that is explicit global knowledge 
and a helpful framework within which to model self-adaptation in an EC system. The populace 
level element of the cultural algorithm be Evolutionary Programming (EP). The worldwide 
knowledge which has been learned because of the population should be expressed with regards 
to both normative and situational knowledge. In this algorithm, first the belief space in addition 
to population space are initialized. Then, the algorithm shall repeat processing for every single 
generation until a termination condition is achieved. People are evaluated making use of the 
performance function. The 2 amounts of Cultural Algorithm communicate by using the 
acceptance function together with influence function. The acceptance function determines 
which folks from the current population is selected to impact the belief space. The selected 
individuals’ experiences that are generalized and applied to modify the existing beliefs when 
look at the belief space through the update function. This new belief is able to be employed to 
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guide and influence the evolutionary process when it comes to next generation. Cultural 
algorithms as explained above comprise of three components. First, there was a population 
component which contains the social population be evolved in addition to mechanisms for the 
reproduction, evaluation, and modification. Second there was a belief space that presents the 
bias which has been acquired because of the population during its problem-solving process. 
The third component is the communications protocol which is used to look for the interaction 
amongst the population and their beliefs. Cultural algorithm is in-depth analysis regarding the 
superiority regarding the evolution that is original on the cornerstone of drawing from the social 
(cultural) evolution theory when look at the social sciences [21]. 
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Figure 2 Cultural Algorithm Framework 


3.3. Genetic Algorithm 


d Genetic Algorithm is characterized by a search technique as described in the Darwin’s 
evolutionist theory which adapts selection mechanism where individuals, that is the 
chromosomes, more adapted of a population are the ones that have more survival chances and 
can be used easily according to changes that occur in its environment. This makes the algorithm 
strong and fast in those situations where the search space is too big and the conventional 
methods become incident. In the genetic algorithm, the problem to be solved is represented by 
a list of parameters which drives an evaluation procedure, called chromosomes or genomes. 
Chromosomes can be defined as simple strings of data and instructions. In the first step of the 
algorithm, such chromosomes are generated randomly or heuristically to form an initial pool of 
possible solutions called first generation pool. The following steps are involved in the Genetic 
algorithm for Face Recognition pre-processing technique. 


Step 1: Firstly, suppose we have a population of N size, with chromosomes generated randomly; 
Step 2: Apply fitness to each chromosome or genomes of population; 


Step 3: Make new chromosomes or genomes through crossings of selected chromosomes of 
this population. (4) Apply recombination and mutation in these chromosomes; 


Step 4: Eliminate old population members, so that there 1s enough space to insert new 
chromosomes, keeping the population with the same N chromosomes; 
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4. RESULT AND DISCUSSION 


The face emotion recognition dataset is taken from the Kaggle repository [22]. The dataset is 
composed of angry, disgust, fear, happy, neutral, sad and surprise emotions. For this paper, 100 
images from each emotion category is considered to evaluate the performance of the pre- 
processing techniques with two classification techniques like ANN, KNN and SVM. 
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Figure 3 Sample human emotions images for analysis 


4.1. Performance Metrics 
Table 1 depicts the performance metrics are used in this research work. 


Table 1 Performance Metrics used in this paper 


Performance Metrics Equation 


PPP DN Ane PN 


Sensitivity 





TP +FN 
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Specificity TN 
TN +FP 


False Positive Rate 1- Specificity 


MissRate _t- Sensitivity 





Table 2 depicts the performance analysis of the pre-processing techniques with KNN and 
CA is compared using the classification techniques with ANN classifier for the classification 
of human emotion recognition with proposed technique. Table 3 presents the performance 
analysis of the pre-processing techniques with KNN and CA is compared using the 
classification techniques with KNN classifier. Table 4 presents the performance analysis of the 
pre-processing technique with KNN and CA is compared using the classification techniques 
with SVM classifier. 

From the table 2, table 3, and table 4 it 1s clear that the Pre-Processing technique with ANN 
performed better in terms of increased Detection Rate, sensitivity, Specificity and reduced error 
rates like FPR, and Miss Rate than the other pre-processing techniques with ANN, KNN and 
SVM classifiers. 


Table 2 Performance analysis of the image pre-processing techniques using ANN classification 


Performance Metrics (in %) Techniques for Pre-Processing of Images 
Genetic Algorithm =| = KNN | CA 
Detection Rate 51.78 56.52 53.69 


Table 3 Performance analysis of the image pre-processing techniques using KNN classification 


Performance Metrics (in %) 


Table 4 Performance analysis of the image pre-processing technique using SVM classification 


Performance Metrics (in %) Techniques for Pre- rocessng of Images 
Genetic Algorithm =| = KNN | CA 
Detection Rate 9202 = a 50.31 











5. CONCLUSION 


Human face recognition and human emotion recognition is a challenging task because of the 
variability of facial expressions, personal appearances, variant poses, and illumination. An 
image pre-processing technique using Genetic Algorithm (GA), Cultural Algorithm (CA) and 
K-Nearest Neighbor (KNN) is proposed to enhance the quality of the image by removing the 
noise, illumination in the face emotion images. From the result obtained by the proposed pre- 
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processing techniques with ANN classification method produced enhanced detection rate, 
sensitivity, specificity for finding the human emotion and it also gave less error rates. 
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